Caries risk assessment using different Cariogram models. A comparative study about concordance in different populations—Adults and children

This methodological survey aimed to verify whether there is concordance among several Cariogram different risk models at different thresholds, comparing both children and adult populations and how each risk/protective factor weight on the overall caries risk profile. Three groups’ data (two in children and one in adults) were obtained from previous studies, while a fourth, in young adults, was ad hoc enrolled. Different caries risk levels were assessed: a) three risk categories with two different thresholds as: “low risk”  =  61–100% or 81–100% chance to avoid caries, “moderate risk”  =  41–60% or 21–80% and “high risk”  =  0–40% or 0–20%, named model 1 and 2; b) four risk categories with two different thresholds as: “low risk”  =  61–100% or 76–100%, “moderate/low risk” = 41–60% or 51–75%; “moderate/high risk” = 21–40% or 26–50% and “high risk” = 0–20% or 0–25%, model 3 and 4; c) five risk categories as: “very low risk”  =  81–100%; “low risk”  =  61–80% “moderate risk” = 41–60%; “high risk” = 21–40% and “very high risk” = 0–20%, model 5. Concordance of the different Cariogram risk categories among the four groups was calculated using Cohen’s kappa. The weight of the association between all Cariogram models toward the Cariogram risk variables was evaluated by ordinal logistic regression models. Considering Cariogram model 1 and 2, Cohen’s Kappa values ranged from 0.40 (SE = 0.07) for the young adult group to 0.71 (SE = 0.05) for the adult one. Cohen’s Kappa values ranged from 0.14 (SE = 0.03 p<0.01) for the adult group to 0.62 (SE = 0.02) for the two groups of children in models 3 and 4. Statistically significant associations were found for all Cariogram risk variables excepting Fluoride program in models 4 and 5 and the overall risk on children’s samples. Caries experience showed a quite variable weight in the different models in both adult groups. In the regression analyses, adult groups’ convergence was not always achievable since variations in associations between caries risk and different risk variables were narrower compared to other samples. Significant differences in caries risk stratification using different thresholds stands out from data analysis; consequently, risk assessments need to be carefully considered due to the risk of misleadingly choosing preventive and research actions.


Introduction
Assessing and classifying individual caries risk is an essential step in the caries diagnosis and decision-making process. Caries risk assessment allows to achieve the best non-invasive and operative options, as well as establishing recall intervals [1,2].
Caries Risk Assessment (CRA) should be an integral part of the treatment regimens, in order to plan an individualized caries prevention strategy. However, routinely the patient caries risk profile is not always determined and, when performed, the risk level is not always linked to the adopted preventive and intervention strategies [3,4].
From the 2000s onwards, a variety of standardized tools were developed and tested in different populations [5,6]. All multifactorial risk tools include a different combination of risk and protective factors, attributing to each variable a different weight in the caries risk calculation [7]. Despite this, the scientific evidence on CRA tools validity remains limited [5,6,8]. Overall, the Cariogram is the only multidimensional tool verified in children and adults in few good quality studies, even if its accuracy level was found to be limited in different age groups [5,9]. Moreover, the Cariogram has been applied in different ways: considering seven to ten risk variables, considering three to five risk categories and finally, considering different thresholds to define each risk category [10][11][12]. These thresholds can lead to a loss of information because the entire range of predicted probabilities is not used, but also to substantial bias because the thresholds can occur as data-driven instead of pre-specified clinical data [9]. In particular, for risk assessment, an inappropriate use of a diagnostic tool can theoretically lead to both clinical and research bias. In fact, an underestimation or overestimation of the risk can lead to the choice of an inadequate preventive/therapeutic regime and/or to an incorrect enrolment of a subject in a study sample.
While the effectiveness of the Cariogram used with a reduced number of factors, mostly excluding salivary factors [13][14][15][16][17], has been validated, while the use of different risk categories considering different ranges of percentages to avoid caries was not investigated. In view of the foregoing, the present methodological study aimed, primary, to verify if there is concordance among the Cariogram caries risk categories at different thresholds in children and adults; and secondly, how much each risk/protective variable weight for the different overall caries risk profile in different age groups.

Samples
Four sample groups were considered in the comparisons. Two were obtained from previous studies in children (available at https://www.karger.com/Article/FullText/334932 and https:// www.tandfonline.com/doi/full/10.1080/00016350902740498, respectively) [13,16], one in adults (available at https://www.scielo.br/j/bor/a/ZqcrsmPRxNJQsYPV5nnNTTS/?lang=en) [17], and the fourth one, which included young adults, was ad hoc enrolled. Unlike the two groups of adults who are two distinct samples in which the risk assessment was carried out one time, the two samples of children are the same group of children whose caries risk was evaluated and re-evaluated after two years. Regarding the published papers [13,16,17], the abstracts can be easily retrieved on Pubmed.
Ethical issues were addressed as follows: the retrospective data analysis of archived samples the IRB of the University of Sassari waived the approval (IRB Sassari 20/04/2020); regarding the ad hoc sample the study proposal was accepted by the Ethical Committee of the University Hospital Milan (Comitato Etico Milano 1 n. 2019/ST/167).
All the surveys from which the data derives, were conducted by the same research group; in particular, two authors of the present survey acted as examiners/benchmarks in all surveys: MGC (A young and A old surveys) and GC (C young and C old surveys as examiner and A young and A old surveys as benchmark).
In children's groups, a reduced Cariogram model (excluding saliva parameters: buffer capacity and saliva secretion rate) was build-up to assess the forecast probabilities of new caries. Two years later, the same sample was re-examined and the caries risk re-calculated. The children samples consisted of 957 subjects aged 7-9 years (C young ) and 862 children aged 9-11 years (C old ).
The adult sample (A old ) included 480 subjects aged 30-45 years. Caries risk was calculated through the full Cariogram model. For the last sample of young adults (A young ), undergraduate second and third-year dental students, School of Dentistry, University of Milan, Italy, were invited to participate. This convenience young adult sample consisted of 86 subjects (mean age 23.37 ± 3.11 years).

Data collection
Data for samples C young , C old and A old , were extracted from the previously published studies. Data collection for sample A young followed the same procedures in the adult population study [17], which included: a standardized high-structured self-administered questionnaire (in Italian and English language S1 and S2 Appendices) [18] collecting data on age, oral hygiene habits (frequency of toothbrushing), dietary habits (daily intake of sugary foods and drinks), fluoride exposure in addition to toothpaste and other remineralizing and antibacterial compounds, frequency of dental check-ups, presence of systemic diseases, and use of medications. Additionally, participants received clinical examinations by two examiners (GB and MGC) to determine caries prevalence at dentinal level, caries experience (Decayed, Missing and Filled Teeth index) [19] and plaque index [14]. The examiners were ad hoc calibrated on 15 subjects reporting a strong Cohen's Kappa scores both for inter-and intra-examiner reliability (0.87/ 0.89 for inter-examiner and 0.85 for intra-examiner reliability) [20][21][22]. Finally, saliva chairside tests were performed following the manufacturer's instructions to evaluate the salivary oral streptococci and Lactobacillus spp. counts (CRT bacteria, Ivoclar Vivadent AG, Schaan, Liechtenstein), and buffer capacity (CRT Buffer, Ivoclar Vivadent AG, Schaan, Liechtenstein). The raw-data file was uploaded as S1 Data.

Risk assessment
Caries risk profiles were calculated using the Cariogram for all subjects in the different groups. Cariogram risk for samples C young and C old was calculated including seven risk/protective variables: caries experience, related diseases, diet content, diet frequency, plaque amount, oral streptococci count and fluoride program. For samples A young and A old , in addition to the previous factors, saliva secretion rate and buffer capacity were also included.

Statistical analysis
Cohen´s Kappa values were calculated to assess the agreement of the Cariogram risk categories among the 4 groups. Kappa statistics were tested using z test at a significance level of α = 0.01. Ordinal logistic regression models were built to evaluate the weight of the associations, via Odds Ratio and 95% Confidence Intervals, of the variables inserted on each Cariogram model toward Cariogram categorizations. Table 2 shows the demographic characteristics and caries figures of all groups. The mean caries experience (DMFT/dmft) ranged from 2.37±4.05 in the C young (including both primary and permanent dentitions) to 3.25±4.15 in the A old , with a D/d component ranging from 1.38 ±2.62 to 0.24±0.72, respectively.

Results
Comparisons among groups and the different risk categories and thresholds are displayed in Table 3. Caries risk profiles were statistically significant different for all age groups (p values ranging from <0.01 to 0.02) when considering three (low, moderate and high) and four (low, moderate/low, moderate/high and high) risk categories. Particularly, Cohen´s Kappa values ranged from 0.71 (SE = 0.05) to 0.40 (SE = 0.07) in the A old and the A young groups, respectively. Children groups showed intermediate values. On the other hand, when four risk categories  A multinomial regression analysis was run, using caries risk categories as dependent variables, to evaluate the weight of the association (OR) between the overall risk measured with the different models and the risk/protective variables considered in each model (Table 4). In the C young and C old groups all considered risk variables were significantly associated to the caries risk profiles, except for fluoride program in models 4 and 5. However, each OR values ranged widely among models. While the weight of caries experience, oral streptococci and fluoride program remained quite stable in different models, diet content, diet frequency and plaque amount showed the widest variability in the weight of the associations. In C young , diet content ranged from OR 2. All other risk variables resulted more strongly associated with risk profiles measured using model 1, with oral streptococci count and fluoride program showing the widest variations among models. Finally, given the low variance across risk variables and the reduced number of subjects in some risk categories, in the A old group for models 2 and 4, the regression analysis was only achieved for model 1, 3 and 5. In this group, variation in weight between caries risk and different risk variables were narrower less broad compared to the above described samples. Fluoride program reached the highest OR in model 1 (3.16 95%CI 1.45 / 6.91) and buffer capacity in model 3 and 5 (OR 7.28 95%CI 0.90 / 34.62, in both models).

Discussion
This study was designed to evaluate the concordance among caries risk profiles evaluated with Cariogram at different risk categories and different thresholds. The study used previous published data (two samples of schoolchildren and one of adults) and original data from a fourth group of young adults. The significant difference in caries risk stratification using different thresholds stands out as the main outcome of the present study.
The Cariogram demonstrates the caries risk graphically and provides a percentage value illustrating the probability an individual has to avoid the development of lesions in the near future. Nevertheless, the weights attributed to the risk factors considered in the Cariogram did not appear to explain caries experience or caries activity in all groups [23].
When assessing the caries risk what is expected to forecast is the chance to develop or not the disease and, in this perspective, only the subsequent incidence of "true" caries can confirm or disprove the initial hypothesis. The Cariogram was initially tested in children considering five risk categories [10]. Afterwards, its validity was demonstrated in elderlies and adults using different Cariogram models [24,25]. Lack of calibration is a common problem of the Cariogram software, as the predictive capacity developed using data from a certain population can decrease when the tool is applied to another population. In these conditions, a recalibration can improve predictive capacity [6].
From a public health perspective, when caries risk is assessed in population samples, it is useful to allocate each subject into a risk category, facilitating the implementation of largescale preventive and management interventions. However, the need to allocate subjects into risk categories for community purposes, as emerges from the findings of the present methodological study, can lead to risk category allocation biases due to the number of risk categories assessed and, above all, to the thresholds used to define each category. The allocation biases were perceived for all considered age groups. It is likely that different groups with different caries prevalence and risk variables could lead to different results. The present study findings might serve as a warning to public health dentists, epidemiologists and researchers on the adoption of the Cariogram models with different thresholds may lead to bias and an incorrect interpretation of the subject's real risk. As suggested in the original Cariogram manual, [26,27] low caries risk is only determined when the "Chance to avoid cavities" is equal to 75% or higher, while a high caries risk is plausible when the 'Chance to avoid cavities' is equal to 25% or lower. No suggestion is given for intermediate values. Nevertheless, in one of the earliest papers published on Cariogram, five risk categories were used (model 5 in the present paper) [27]. From a community or research perspective, an erroneous risk categorization could lead to unnecessary or inadequate preventive interventions [5]. This bias is confirmed by the analysis of the weight of the association between the risk/protective variables and the overall caries risk evaluated through the different five Cariogram models. The background risk factors (i.e. related disease, diet content, diet frequency, plaque amounts, oral streptococci, fluoride program, saliva secretion, buffer capacity) had a very different association with the caries risk measured with each model. Hence, the association between the caries risk and the background variables might be miscalculated and misunderstood. For example, the variable plaque amount assumes within the same sample an extremely high weight in some models and a reduced one in others. Also, the role of fluoride program is emblematic; it is not even associated to caries risk in some models (both in children and adults), while strongly associated in some of the adult models, and becomes weaker in others.
On the one hand, the Cariogram assessment at the individual level, allows to graphically evaluate which factors are the heaviest contributors to an individual caries risk, allowing to intervene on them. On the other hand, as previously reported, the graphical advantages of the Cariogram are practically unattainable at a community level, leading to non-effective interventions from a clinical and/or cost / benefit point of view [28,29].
A major limitation of the present paper is undeniably the small number of participants in the sample A young . The small number of participants is unfortunately related to the historical moment, COVID-19 pandemic, in which the study was carried out. As the lockdown was declared by the Italian Government, the study was halted with the hope to resume in September; unfortunately, the conditions related to the pandemic made it impossible, resulting so in a small sample size. Although with this factual limitation, the authors believe that the sample can provide proper support in the paper. In this sample, caries experience showed a quite variable weight in the different models, reaching the highest value in model 3, allowing some generalisation of the results.
Some strengths have to be considered. First of all, this is the first study that compares different models of the Cariogram in different populations. In fact, it is known that the tool does not work as well in all population groups [5,9]. Furthermore, the fact of having tested different models both in samples of children and adults of different ages gives value to the results of this comparative paper. Finally, having assessed for each model and in each population group the strength of association of each risk factor with the overall caries risk, is a new and little explored aspect that has allowed to verify how much each factor play a different role in the different models and samples.

Conclusions
The outcomes of this study might allow to conclude that the caries risk assessment is modified when the Cariogram thresholds vary. The interpretation of caries risk profiles needs to be carefully considered in dental public health, given the risk of misleading the choice of both preventive and management actions, as well as the wrong selection of subjects in specific caries risk categories for research purposes.
Supporting information S1 Checklist. STROBE statement-checklist of items that should be included in reports of cross-sectional studies.